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Abstract 

In  the  classical  sense,  data  fusion  can  be  viewed  as  a  one-dimensional  entity  having  five 
distinct  levels.  However,  this  view  does  not  convey  the  multi-dimensional  aspect  of  data 
fusion.  This  paper  argues  that  data  fusion  is  not  one-dimensional,  but  rather  a  three- 
dimensional  entity.  These  three  attributes  are  sensor  fusion,  system  fusion  and 
information  fusion.  Sensor  fusion  can  be  thought  of  as  taking  the  raw  sensor  data  and 
fusing  it  together  so  it  seems  to  have  come  from  a  single  sensor.  System  fusion  can  be 
thought  of  as  combining  the  output  of  various  heterogeneous  systems  together  into  a 
single  fused  output.  Information  fusion  can  be  thought  of  as  taking  information  gathered 
from  various  sources  and  fusing  it  into  a  single  output.  This  paper  provides  an  overview 
and  discussion  of  this  three-dimensional  perspective  of  data  fusion  in  order  to  illustrate 
its  multi-faceted  capabilities  and  applications.  For  each  of  the  three  dimensions,  a 
definition  and  possible  application,  along  with  a  discussion  and  comparison  to  the 
classical  Level-0  to  Level-4  levels  of  data  fusion,  is  presented.  Finally,  this  new  data 
fusion  “cube”  is  offered  for  consideration  in  which  each  axis  (Sensor,  System, 
Information)  has  a  corresponding  relationship  to  the  classical  aspects  of  data  fusion. 

Introduction 

The  importance  of  fusion  is  quite  evident.  The  Commander  wants  an  all-source  picture  of 
the  battlespace  to  provide  real-time  situational  awareness.  However,  today  all  the 
Commander  gets  is  an  extremely  large  number  of  uncorrelated  reports  that  further 
contribute  to  what  has  become  known  as  “the  fog  of  war”.  Fusion  is  a  methodology  to 
merge  all  available  information  to  provide  all-source  consistent  actionable  information  or 
knowledge  throughout  the  battlespace  to  the  Commander  so  that  rapid,  intelligent 
decisions  can  be  made.  To  the  Commander,  this  merging  of  information  and  data  sources, 
as  well  as  refinement  of  information,  appears  to  come  from  one  source. 

Historically,  the  US  military  has  spent  enormous  amounts  of  resources  in  building 
aerospace  platforms,  (probably  because  more  information  is  “better”  information)  but, 
little  has  been  spent  dealing  with  the  corresponding  use  of  the  data  to  be  gathered;  e.g., 
fusion,  decision  making  and  ultimately  Command  and  Control  (C2).  A  prime  example  is 
hyperspectral  sensors.  A  significant  amount  of  resources  have  gone  into  the  design  of  the 
space  segment;  however,  little  has  gone  into  the  Tasking-Processing-Exploitation- 
Dissemination  (TPED)  portion.  For  a  3500-band  hyperspectral  sensor,  it  is  unrealistic  to 
think  that  all  the  processing  will  be  accomplished  on  the  ground.  Some  amount  of 
processing,  fusion  and  exploitation  must  be  accomplished  on-board  the  space  vehicle. 

Recently,  however,  there  has  been  a  turnaround  and  there  have  been  significant  resources 
allocated  to  developing  technology  to  support  data  fusion  to  allow  the  rapid  use  of 
extremely  large  amounts  of  information  from  diverse  sources.  For  example,  the  Air  Force 
Research  Laboratory’s  Information  Directorate  has  an  Advanced  Technology 
Development  initiative  known  as  “Adaptive  Sensor  Fusion  -  ASF”.  The  ASF  Program  is 
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building  an  open,  standards-based  architecture  for  information  fusion.  The  philosophy  of 
the  open,  plug  and  play  fusion  architecture  is  to  provide  a  system  that  provides  standard 
data  and  application  interfaces  to  maximize  interoperability  among  different  fusion 
approaches  and  systems.  The  fusion  architecture  includes  an  adaptive  fusion  manager 
that  optimizes  the  performance  of  the  selected  confederation  of  fusion  engines.  The 
objectives  are  the  optimization  and  control  of  the  fusion  process  to  foster  an  effective  and 
complete  use  of  all  available  sensor  data.  The  Adaptive  Sensor  Fusion  System  will  have 
the  ability  to  use  different  fusion  parameters,  models  and  configurations,  optimized 
according  to  specified  user  needs,  changes  in  the  situation,  and  available  fusion  engines. 
This  is  a  one-dimensional  approach  to  fusion. 

“One-dimensional”  Data  Fusion  Overview 

Figure  1  below  represents  the  classical  (or  one-dimensional)  model  of  fusion  breaking  it 
down  into  five  levels. 


Figure  1 :  Data  Fusion  Levels 


DSP 


SBIRS 


■  Intel  Sources 
■  Air  Surveillance 
■  Surface 
Surveillance 
Space  Surveillance 


Level  0 
Processing 

Sub-object  Data 
Association  & 
Estimation 


DATA  FUSION  DOMAIN 


Level  1 
Processing 

Object 

Refinement 


Level  2 
Processing 


Level  3 
Processing 

Impact 

Assessment 


Level  4 
Processing 


Data  Base 

Management  System 


The  five  levels  can  be  defined  as  follows: 

a)  Level  0  —  Sub-Object  Data  Association  &  Estimation:  pixel/signal  level  data 
association  and  characterization; 

b)  Level  1  —  Object  Refinement:  observation-to-track  association,  continuous  state 
estimation  (e.g.  kinematics)  and  discrete  state  estimation  (e.g.  target  type  and  ID)  and 
prediction; 

c)  Level  2  —  Situation  Refinement:  object  clustering  and  relational  analysis,  to  include 
force  structure  and  cross  force  relations,  communications,  physical  context,  etc.; 

d)  Level  3  —  Impact  Assessment:  includes  threat  refinement,  threat  intent  estimation, 
event  prediction^  consequence  prediction,  susceptibility  and  vulnerability  assessment, 
and; 

e)  Level  4:  Process  Refinement:  adaptive  search  and  processing  (an  element  of  resource 
management) 

This  is  the  classical  approach  to  the  topology  of  fusion.  It  promotes  a  data  ontology  technology 
that  will  result  in  a  plug-and-play  architecture  that  can  provide  a  standard  for  fusion.  However,  this 
view  can  be  expanded. 


“Three-dimensional”  Data  Fusion:  Proposed  Perspective 

Fusion  takes  on  different  attributes  depending  on  what  type  of  fusion  is  being  performed. 
Fusion  can  be  thought  of  in  three-dimensions:  sensor,  system  and  information.  They  are 
briefly  discussed  as: 

a)  Dimension  1:  Sensor  Fusion 


Senor  fusion  can  be  thought  of  as  fusing  raw  data  from  a  multitude  of  sensors  together  to 
create  a  single  sensor  output.  This  is  like  making  each  sensor  a  mode  or  component  of  a 
larger  integrated  sensor.  Figure  2  portrays  a  cluster  of  three  microsats.  Each  microsat 
would  have  a  sensor.  Integrating  the  three  sensors  together  from  the  three  microsats  into  a 
single  “sensor”  creating  a  single  target  pixel  would  be  an  example  of  sensor  fusion. 


Figure  2:  Sensor  Fusion 


b)  Dimension  2:  System  Fusion 

The  second  dimension  of  fusion  can  be  thought  of  as  fusing  two  different  sensors 
located  on  two  different  systems  (e.g.,  EO-EO,  SAR-SAR,  IR-IR,  EO-SAR,  etc.)  to 
create  a  single  integrated  output.  Figure  3  illustrates  the  fusion  of  three  different 
systems  to  create  a  single  target  pixel.  Note  that  this  is  not  just  an  overlay  of  three 
ellipses,  but  also  a  true  fusion  of  the  outputs  of  the  three  systems.  The  resultant  error 
ellipse  for  fusion  would  be  smaller  than  the  overlays  of  the  three  ellipses. 


Figure  3:  System  Fusion 


c)  Dimension  3:  Information  Fusion 

The  third  dimension  of  fusion  is  information  fusion.  This  is  the  fusion  of  information 
that  is  gathered  from  a  particular  system;  e.g.,  human  intelligence  (HUMINT), 
imagery  intelligence  (IMINT)  or  signals  intelligence  (SIGINT)  and  converted  into 
some  general  knowledge.  Figure  4  illustrates  some  of  the  sources  for  information 
fusion,  and  the  resultant  attribute  of  decision  quality  information. 


Figure  4:  Information  Fusion 


Fusion  Dimension  Levels 


Combining  the  three  dimensions  of  fusion  (sensor,  system  and  information)  makes  up  the 
overall  data  fusion  cube.  However,  each  of  the  levels  means  different  things  regarding  the 
three  dimensions.  For  example: 


a)  Sensor  Fusion  Dimension/Levels 

Sensor  fusion  deals  primarily  with  the  first  two  levels  of  the  classical  model,  namely  Level-0  and 
Level-1  (as  shown  in  Figure  1 .  These  two  levels  technologically  relatively  mature.  However,  the 
difficulty  lies  in  trying  to  track  targets  in  a  dense  target  environment  with  highly  maneuverable 
targets.  The  sensor  fusion  technologies  applicable  to  Level-0  and  Level-1  are  locating  sensors  on 
multi-platforms  and  providing  precision  tracking  of  those  targets. 

b)  System  Fusion  Dimension/Levels 

System  fusion  covers  not  only  Level-0  and  Level-1,  but  since  this  also  incorporates  other 
system’s  sensors  (e.g.,  EO,  SAR,  IR)  located  on  additional  platforms,  it  also  incorporates  Level-2. 
Although  Level’s  0  and  1  are  relatively  mature,  Level-2  is  immature  from  a  technology 
perspective. 

There  is  a  lack  of  automated  reasoning  techniques,  cognitive  models  and  the  doctrinal  foundation 
is  not  well  defined.  The  system  fusion  technologies  applicable  to  Level-2  must  provide  robust 
techniques  to  solve  situation/threat  refinement  along  with  providing  predictive  analysis 
methodologies. 

c)  Information  Fusion  Dimension/Levels 

Information  fusion  deals  primarily  with  the  upper  three  levels  of  the  classical  fusion  hierarchy, 
namely  Levels-2  through  4.  It  is  in  these  three  levels  that  decision  support  technologies  play 
a  major  part,  especially  with  the  prediction  and  assessment  in  Level-3.  Levels-3  and  4  are 
very  immature  from  a  technology  perspective.  The  technology  areas  deal  with  generic 
architecture  and  techniques  for  fusion  process  control,  predictive  battlespace  awareness, 
plug-N-play  fusion  algorithms,  adaptive  fusion  management  and  situation  refinement. 

From  a  technology  maturity  point-of-view,  the  state-of-the-art  in  data  fusion  is  at  Level- 
0-1-2.  Additional  research  needs  to  be  done  in  the  areas  of: 

a)  Assessment  of  the  overall  information  fusion  space  with  statistical  and  learning 
techniques  to  identify  weak  points  within  the  overall  fusion  space. 

b)  Investigate  and  evaluate  use  of  local  fusion  models  embodied  in  dynamically 
created,  distributed  intelligent  agents, 

c)  Investigate  operational  plan  recognition  and  plan  inference,  based  on  multi¬ 
level  fused  data;  and, 

d)  Development  of  measures  of  performance/measures  of  effectiveness 
(MOP/MOE)  to  be  able  to  evaluate  the  upper  three  levels  of  fusion. 


Space  Application 


The  three  dimensions  of  fusion  occur  naturally  at  three  different  levels  within  the  total 
system  architecture.  Figure  5  illustrates  the  placement  of  the  three-dimension  of  fusion  in 
a  generic  space  system.  Each  of  the  three  dimensions  (Sensor,  System,  Information)  is 
represented. 


Figure  5:  Generic  Space  Architecture 


ble.  In  Figure  5,  this  is  on-board  either  the  space  or  air  vehicle. 


System  fusion  typically  occurs  within  the  Control  Center,  since  this  is  where  the  raw  data 
processing  normally  occurs.  Naturally,  taking  advantage  of  advanced  technologies,  this 
processing  could  be  performed  on-board  the  vehicle.  If  system  fusion  is  to  occur,  then  a 
communications  path  needs  to  be  established  between  the  two  systems  so  that  the  fusion 
process  can  take  place. 


Information  fusion  naturally  occurs  at  the  user  point  since  this  is  where  processed 
information  comes  together.  As  discussed  above,  this  can  be  accomplished  on-board  the 
vehicles,  but  then  the  appropriate  fusion  algorithms  must  be  located  on-board  each  of  the 
vehicles,  so  that  a  complete  process  can  be  performed. 


Figure  6:  Multi-dimensional  Data 
Fusion  Cube 

Summary 

This  paper  illustrated  that  data  fusion 
can  be  thought  of  as  a  three-dimension 
“cube”.  Each  dimension  (senor,  system 
and  information)  has  unique  properties 
and  provides  something  different  to  each 
of  the  five  levels  of  fusion. 

Figure  6  illustrates  the  multi¬ 
dimensional  data  fusion  “cube”. 


This  paper  also  proposes  that  this  Cube” 
topology  be  used  when  approaching  the 
fusion  problem. 


Figure  6  illustrates  the  relationships  of 
the  fusion  “cube”  as  it  fits  into  the 
Situational  Awareness  of  the 
Battlespace. 
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